Fox News Flash top headlines are here. Check out what's clicking on Foxnews.com. We are in the midst of an artificial intelligence (AI)-driven industrial revolution. From self-driving cars to medical diagnostics to next-generation defense and homeland security capabilities, AI is reshaping nearly every industry. As the U.S. races to maintain its global leadership in AI, much of the conversation revolves around natural language processing, the reshoring of the semiconductor supply chain and powering data centers.

For hundreds of years, any organisation that needed to store information relied on one tried and tested technology: paper. But since the advent of computing and digital data storage, more and more data has been captured and stored electronically in digital archives. But now organisations need to retain archived data for longer – for business and regulatory reasons – can storage technology keep up? No computer system is older than 80 years, but there are industries that face the prospect of archiving data for 100 years or more. And, with the operating lifespan of a standard hard drive at just three to five years, IT departments need to know how to store data for future generations: so-called indefinite storage.

Large, monolithic datacentres at the heart of enterprises could give way to hundreds or thousands of smaller data stores and devices, each with their own storage capacity. This driver for this is organisations moving their processes to the business "edge". Edge computing is no longer simply about putting some local storage into a remote or branch office (ROBO). Rather, it is being driven by the internet of things (IoT), smart devices and sensors, and technologies such as autonomous cars. All these technologies increasingly need their own local edge data storage.

A byte of data has been stored in a number of different ways through the years as newer, better, and faster storage media are introduced. A byte is a unit of digital information that most commonly refers to eight bits. A bit is a unit of information that can be expressed as 0 or 1, representing a logical state. Let's take a brief walk down memory lane to learn about the origins of bits and bytes. Going back in time to Babbage's Analytical Engine, you can see that a bit was stored as the position of a mechanical gear or lever. In the case of paper cards, a bit was stored as the presence or absence of a hole in the card at a specific place. For magnetic storage devices, such as tapes and disks, a bit is represented by the polarity of a certain area of the magnetic film. In modern DRAM (dynamic random-access memory), a bit is often represented as two levels of electrical charge stored in a capacitor, a device that stores electrical energy in an electric field. In June 1956, Werner Buchholz coined the word byte to refer to a group of bits used to encode a single character of text. Let's address character encoding, starting with ASCII (American Standard Code for Information Interchange). ASCII was based on the English alphabet; therefore, every letter, digit, and symbol (a-z, A-Z, 0-9,, -, /, ",!, among others) were represented as a seven-bit integer between 32 and 127. To support other languages, Unicode extended ASCII so that each character is represented as a code-point, or character; for example, a lowercase j is U 006A, where U stands for Unicode followed by a hexadecimal number. UTF-8 is the standard for representing characters as eight bits, allowing every code-point from 0 to 127 to be stored in a single byte. This is fine for English characters, but other languages often have characters that are expressed as two or more bytes.

When it comes to data, AI is like Pac-Man. Hard disk drives, NAS, conventional data center and cloud-based storage schemes can't sate AI's voracious appetite for speed and capacity, especially for real time. Playing the game today requires a fundamental rethinking of storage as a foundation of machine learning, deep learning, image processing, and neural network success. "AI and Big Data are dominating every aspect of decision-making and operations," says Jeff Denworth, vice president of products and co-founder at Vast Data, a provider of all-flash storage and services. "The need for vast amounts of fast data are rendering the traditional storage pyramid obsolete. Applying new thinking to many of the toughest problems helps simplify the storage and access of huge reserves of data, in real time, leading to insights that were not possible before."

Microsoft and Warner Bros. have collaborated to successfully store and retrieve the entire 1978 iconic "Superman" movie on a piece of glass roughly the size of a drink coaster, 75 by 75 by 2 millimeters thick. It was the first proof of concept test for Project Silica, a Microsoft Research project that uses recent discoveries in ultrafast laser optics and artificial intelligence to store data in quartz glass. Machine learning algorithms read the data back by decoding images and patterns that are created as polarized light shines through the glass. The hard silica glass can withstand being boiled in hot water, baked in an oven, microwaved, flooded, scoured, demagnetized and other environmental threats that can destroy priceless historic archives or cultural treasures if things go wrong. It represents an investment by Microsoft Azure to develop storage technologies built specifically for cloud computing patterns, rather than relying on storage media designed to work in computers or other scenarios.

We live in a data-driven world. Successful, leading companies have mastered and operationalized the process of extracting insight and intelligence from all the data collected continuously. The use of data has brought on a sea change in business models, with AI being the primary technique used to distill all this data into actionable insight. ML/DL depends on training and inference, both of which require fast execution with large data sets flowing smoothly through the pipeline. These algorithms perform better and become more accurate as the training data sets grow.

When comes to machine learning, data is certainly the new oil. The processes for managing the lifecycle of datasets are some of the most challenging elements of large scale machine learning solutions. Data ingestion, indexing, search, annotation, discovery are some of the aspects required to maintain high quality datasets. The complexity of these challenges increase linearly with the size and number of the target datasets. While it is relatively easy to manage training datasets for a single machine learning model, scaling that process across thousands of dataset and hundreds of models can become nothing short of a nightmare. Some of the companies at the forefront of machine learning innovation such as LinkedIn, Uber, Netflix, Airbnb or Lyft have certainly experienced the magnitude of this challenge and they have built specific solutions to address it.

Within the enterprise data center, today's state of the art is the All-Flash Array, which depends upon non-volatile Flash storage to provide high performance and durability, as compared to the spinning disk technology that preceded it. Until recently, however, Flash arrays were decidedly more expensive than hard drives, prompting enterprises to implement a mix of different storage technologies for different purposes. At the high end, Flash supports high performance computing (HPC) and certain mission-critical tasks that require real-time processing of data – what we call'hot data.' For top performance, hot data require expensive network protocols like InfiniBand or similarly costly storage-area networks (SANs) that depend upon Fibre Channel networking technology. In the middle are'warm data' on hard drives that leverage earlier spinning disk technology. Due to their moving parts, such disks wear out with annoying frequency.

When it comes to cloud services and software-as-a-service (SaaS), we're all familiar with the usual players. During my recent conversation with Aaron Levie, the CEO of Box, or the company's "chief magician," as his title used to read, he presented specific advice for CIOs during this period of digital transformation in change. You can catch part one of our conversation and watch the entire video, which was episode 278 of the CXOTalk series of conversations with the world's top innovators. View the video excerpt above and read Aaron's edited comments below for insightful thoughts directed toward chief information officers. The bottom line: Being adaptable is the key to maintaining CIO relevance to the business.